AMENDMENTS TO THE CLAIMS 

1-8. (Cancelled) 

9. (Currently Amended) An ingredient cooking-operation recognition system comprising: 

a sensing means for acquiring observation data which includes at least either of an image 
around the hands of a person who performs a cooking operation and an environment sound that is 
produced by the cooking operation; 

a feature-quantity template in which various cooking operations that are predetermined 
for various ingredients are stored together with a template certainty factor that is a certainty 
factor which is predetermined for each cooking operation of the ingredients; 

a cooking-flow creating means for, based on the observation dat a which is acquir e d by 
the sensing means , calculating an observation certainty factor which indicates the certainty factor 
of at least either of an ingredient that is cooked by the person and the cooking operation of the 
ingredient, and based on this-the observation certainty factor, creating a cooking flow for a dish 
which is prepared by the person; 

a primary certainty-factor calculating means fo r, based on th e template certainty factor 
and th e observation certainty factor, calculating a primary certainty factor which is a certainty 
factor for each of obj e cts object that make -makes u p the cooking flow which is cr e at e d by th e 
cooking flow cr e ating m e ans based on the template certainty factor and the observation certainty 
factor ; and 

a cooking-operation recognizing means for recognizing the ingredient and the cooking 
operation that correspond to the observation dat a which is acquired by the s e nsing means , based 
on the primary certainty facto r which is calculated by th e primary certainty factor calculating 
means ; and 

a guidance means for providing guidance to the person regarding the cooking recipe 
based on the ingredient and the cooking operation recognized by the cooking-operation 
recognizing means, wherein the person performs the cooking operation in accordance with the 
guidance . 

10. (Currently Amended) The ingredient cooking-operation recognition system according to 
claim 9, further comprising: 
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a cooking-flow database which stores cooking flows which are created in advance of 
cooking operations for various dishes; and 

a relevance-factor calculating means for, using mapping, calculating a relevance factor 
that indicates which of the cooking flows that are stored in the cooking-flow database is closest 
to the cooking flow which is created by the cooking-flow creating means, 

wherein the cooking-operation recognizing means calculates a final certainty factor based 
on the relevance factor which is calculated by the relevance-factor calculating means and the 
primary certainty factor, updates the feature-quantity template by considering the final certainty 
factor as a new template certainty factor, and based on the feature-quantity template, recognizes 
an ingredient and a cooking operation. 

11. (Currently Amended) The ingredient cooking-operation recognition system according to 
claim 9, further comprising: 

a table storing means for storing at least any one of : 

an ingredient table which is formed by an ingredient record that has a field 
where data that indicates an ingredient is stored and a field where various kinds of 
data on the ingredient are stored together with a-the template certainty factor that 
is predetermined for the data; 

a cooked-food table which is formed by a cooked-food record that has a 
field where data that indicates a cooking operation is stored and a field where 
various kinds of data on a change in an ingredient by the cooking operation are 
stored together with a-the template certainty factor that is predetermined for the 
data; and 

a cooking-operation table which is formed by a cooking-operation record 
that has a field where data that indicates a cooking operation is stored and a field 
where various kinds of data on an environment sound that is produced by the 
cooking operation are stored together with a-the template certainty factor that is 
predetermined for the data^ 
a cooking-recipe database which stores data that indicates an ingredient which is used for 
various cooking recipes and data that indicates a cooking operation for the ingredient; 
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an extracting means for extracting all kinds of ingredients and all kinds of cooking 
operations which are stored in the cooking-recipe database; and 

a feature-quantity template creating means for creating, as a feature-quantity template, a 
table which includes fields defined by ; ras-items including each ingredient that is extracted by 
the extracting means and each cooking operation that is extracted by the extracting means, and 

the f e atur e quantity t e mplat e cr e ating m e ans writing for specifying , in an optional field of 

the fields which make up the feature-quantity template, as th e template certainty factor - the 
greatest certainty factor of th e t e mplat e certainty factors which ar e stored in each fi e ld of at l e ast 



ingredient that corresponds to the optional field, the cooked-food record of the cooked-food table 
that stores the data which indicates the cooking operation that corresponds to the optional field 
and the cooking-operation record of the cooking-operation table that stores the data which 
indicates the cooking operation that corresponds to the optional field , and writing the greatest 
template certainty factor among the template certainty factors which are stored in each field of 
the specified records in the optional field . 

12. (Currently Amended) The ingredient cooking-operation recognition system according to 

the sensing means has a thermal camera and an optical camera; and 

an image which is acquired by the sensing means includes a thermal image which is 

photographed by the thermal camera and an optical image which is photographed by the optical 

camera. 

13. (Currently Amended) The ingredient cooking-operation recognition system according to 
claim 12, characteriz e d in that wherein the cooking-flow creating means: 

removes a background region from the optical image by executing a background 
difference proc e ssing p rocess on the optical image; 

removes, from the thermal image, as a background region, a region where the temperature 
is equal to, or higher than, a predetermined value; 

calculates an ingredient region which indicates an ingredient by multiplying the thermal 




-the ingredient record of the ingredient table that stores the data which indicates the 
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image whose background region is removed and the optical image whose background region is 
removed; and 

calculates an observation certainty factor based on this-the ingredient region. 

14. (Currently Amended) The ingredient cooking-operation recognition system according to 
claim 9, characteriz e d in that wherein the cooking-flow creating means obtains a histogram of hue 
and a histogram of saturation from an image which is acquired by the sensing means, and 
estimates an ingredient by obtaining a correlation between these-the histograms and a color- 
feature template which is predetermined for each ingredient. 

15. (Currently Amended) The ingredient cooking-operation recognition system according to 

the relevance-factor calculating means estimates a cooking recipe based on a relevance 
factor; and 

a -the g uidance means provides guidance is furth e r provided for giving, to the 
pefsef frperson guidance on r egarding the cooking recipe which is estimated by th e rel e vanc e- 
factor calculating m e ans . 

16. (Currently Amended) A computer-readable recording medium which is -has r ecorded with 
thereon an ingredient cooking-operation recognition program , charact e rized by allowing for 
causing a computer to function at least as: 

a sensing means for acquiring observation data which includes at least either of an image 
around the hands of a person who performs a cooking operation and an environment sound that is 
produced by the cooking operation; 

a feature-quantity template in which various cooking operations that are predetermined 
for various ingredients are stored together with a template certainty factor that is a certainty 
factor which is predetermined for each cooking operation of the ingredients; 

a cooking-flow creating means for, based on the observation dat a which is acquir e d by 
the sensing m e ans , calculating an observation certainty factor which indicates the certainty factor 
of at least either of an ingredient that is cooked by the person and the cooking operation of the 
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ingredient, and based on this-the observation certainty factor, creating a cooking flow for a dish 
which is prepared by the person; 

a primary certainty-factor calculating means fo r, bas e d on th e templat e c e rtainty factor 
and the observation certainty factor, calculating a primary certainty factor which is a certainty 
factor for each of obiects object that make -makes u p the cooking flow which is created by the 
cooking-flow cr e ating m e ans based on the template certainty factor and the observation certainty 
factor ; and 

a cooking-operation recognizing means for recognizing the ingredient and the cooking 
operation that correspond to the observation data which is acquir e d by th e sending m e ans , based 
on the primary certainly facto r which is calculated by th e primary certainty factor calculating 
mean s; and 

a guidance means for providing guidance to the person regarding the cooking recipe 
based on the ingredient and the cooking operation recognized by the cooking-operation 
recognizing means, wherein the person performs the cooking operation in accordance with the 
guidance . 



